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Research  Study  Overview 

•  Purpose  of  the  Study:  The  purpose  of  this  qualitative  research  study  was 
twofold.  First  was  to  provide  PMs  with  a  simulation  tool  to  assist  Program 
Offices  in  evaluating  the  relative  merits  and  risks  of  utilizing  NAWCAD  and 
commercial  OEMs  for  various  LSI/LCI  roles  and  responsibilities.  Second, 
to  understand  where  the  Government  (NAWCAD)  lacks  competitiveness 
on  the  capability  to  perform  LSI  (identify  the  gaps). 


•  Problem  Addressed:  Comparisons  for  sourcing  are  complex  and  require 
extensive  decision  support  analysis  to  understand  the  solution  space  and 
foresight  between  Original  Equipment  Manufacturers  (OEMs)  and 
Government  entities  performing  the  Lead  System  Integrator  (LSI)  role. 


•  Significance  of  the  Study:  This  research  study  will  help  Program  Offices 
understand  their  most  effective  cost,  schedule  and  performance-based 
solutions  when  faced  with  an  acquisition  requirement. 
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Research  Question 

The  research  focused  on  the  question: 


>How  can  Program  Managers  best  determine 
their  acquisition  source  solutions  in  a  timely 
and  cost  effective  manner? 

NAWCAD  wants  to  ensure  that  the  Program  Offices  that 
it  supports  derive  the  best  value  for  their  resources. 
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Methodology 

•  Dialogue  decision  process  was  used  that  focuses  on  a  structured 
dialogue  between  the  Decision  Board  and  Project  Team 


•  Develop  a  simulation  tool  to  assist  Program  Offices  in  evaluating  the 
relative  risks  of  utilizing  NAWCAD  and  commercial  OEMs  for  various 
LSI  roles  and  responsibilities 


•  Interview  Eleven  NAVAIR  Program  Managers  (PM)  and  sixteen 
Competency  Leads  ,  and  other  stakeholders  were  surveyed  online. 
Risk  drivers  were  codified  into  a  model  based  on  lessons  learned 
from  PM's  experience 


•  Develop  and  demonstrate  Prototype  LSI  directional  tool 


•  Conduct  Four  Beta  tests  on  NAVAIR  programs  to  stress  the  model's 
capability  and  to  identify  potential  improvements 
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Courses  of  Action  Continuum 


Contractor  LSI  Paradigm  Navy  LSI 


The  PM  or  Integrated  Program  Team  (IPT)  can  evaluate  potential 
CoAs  for  any  combination  of  government  or  industry  LSI  activity  at 
the  system,  sub-system  or  component  level. 
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Risk  Drivers  Codified  in  Prototype 


Technical  complexity 
(TRL) 

Re-use  component  or 
software  level 

System  complexity 

—  Dimension  1  -  Product  & 
Context  Complexity 
(Functions) 

—  Dimension  2  -  Interface 
Complexity  &  Information 
Diversity 

Program  complexity 

-  Customer  Complexity 

-  Supplier  complexity 


Contract  type 
Contract  award  time 
Integration  readiness  (IRL) 
Proprietary  data  rights 
Competency  of  personnel 
Availability  of  personnel 
Facility  readiness 
Security  clearance  status 
Pass  through  cost 
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Beta  Project  Objectives 


S  Model  functionality  in  an  operational  environment  with  the 
Program  Management  office  in  the  acquisition  process 

•S  Incorporate  lessons  learned  from  the  Beta  programs  into 
architectural  changes  to  the  model 

•S  Identify  and  modify  model  reporting  and  graphics  required  for 
PMA  use  in  the  acquisition  process 

•S  Demonstrate  directional  merits  of  various  courses  of  action 
regarding  roles  and  responsibilities  of  executing  entities 
enabling  improved  decision  making  by  NAVAIR  leadership  and 
PMs 
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Findings 

•  The  model  quantitatively  demonstrated  relatively  low 
risk  for  Government  LSI  approaches  for  one  of  the  PMA 
led  efforts  in  which  known  contributions  to  risk  were 
accurately  predicted. 

•  The  model  was  used  in  conversation  with  other  PMAs 
to  quantitatively  demonstrate  PMA  success  and 
compare  government  approaches  to  industry 
alternatives. 


•  The  model  demonstrated  the  ability  to  execute 

multiple  CoAs  allowing  for  "what-if"  CoA  assessments 
prior  to  execution  and  to  assess  the  relative  risk  of 
each  proposed  approach. 
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Findings  (Cont) 

•  The  structured  process  of  discussing  and  inputting  the 
various  options  also  led  to  better  overall  IPT  understanding 
of  the  risks  associated  with  each  approach. 

•  The  PMA  should  conduct  the  assessments  as  a  group  or 
team  in  a  disciplined  manner  with  the  aid  of  the  LSI/LCI 
directional  tool.  It  is  critical  to  have  the  right  people  in  the 
room  during  the  assessment.  Key  SMEs  include  the  PM, 
Technical  experts,  Acquisition  expert,  AIR  4.2  Should  Cost 
expert,  IPT  Lead  and  a  Supply  Chain  expert. 


•  The  key  is  the  learning  that  occurs  across  the  entire  IPT  as 
CoAs  are  run,  iterated  and  evaluated. 
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Conclusions 


•  The  concept  of  providing  a  useful  tool  to  the  PMA  for 
programmatic  decisions  that  also  allows  NAWCAD  to 
aggregate  data  on  NAWC  capabilities,  people  and  facilities 
proved  to  be  extremely  insightful. 


•  The  model  delivered  LSI  risk  assessments  as  intended,  with 
useful  feedback  to  the  PMA  and  NAWC  in  the  Beta  phase. 

•  The  PMA  response  in  the  Beta  phase  was  positive  beyond 
intended  LSI  decisions,  and  the  feedback  indicated  value  for 
programmatic  "what-if"  analysis  for  other  PMA  decisions 
such  as  funding  drills,  sources  selection,  etc. 


5/19/2015 


Ted  Delbo 


10 


Recommendations 

•  It  is  recommended  data  generated  by  this  tool  provide 
a  basis  for  understanding  the  workforce  skill  set 
requirements  gaps  across  both  the  PMA  and  NAWC. 

•  It  is  recommended  to  aggregate  data  to  help  NAWC  to 
establish  hiring,  training  and  knowledge  management 
goals  to  meet  its  customers'  requirements. 

•  It  is  recommended  organic  integration  capabilities 
provided  by  NAWC  be  improved  to  better  support  the 
PMA. 

•  It  is  recommended  that  a  Version  1.2  be  developed 
that  provides  a  Web  Based  Programmatic  Decision 
Tool. 


5/19/2015 


Ted  Delbo 


11 


Concluding  Thoughts 

•  The  identification,  development  and  implementation 
of  the  Programmatic  Decision  Tool  for  LSI  decisions, 
followed  a  Dialogue  Decision  Process 


•  It  provides  the  PMA  and  NAWC  with  a  tool  necessary 
for  adapting  efficiently  and  effectively  in  a  VUCA 
environment  (VUCA  stands  for  volatility,  uncertainty, 
complexity,  and  ambiguity)  by  facilitating  collaborative 
decision  making 


•  This  methodology  provides  a  quantifiable  rationale  to 
provide  the  Warfighter  and  taxpayer  with  the  best 
value  solution  for  LSI  and  other  programmatic 
decisions 
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Back-Up 
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